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SHORT COMMUNICATION 

Assessing the construct validity of five nutrient profiling systems 
using diet modeling with linear programming 

E Clerfeuille 1 , F Vieux 2 ' 3 ' 4 , A Lluch 5 , N Darmon 2 ' 3 ' 4 and N Rolf-Pedersen 5 

Nutrient profiling classifies individual food products according to their nutrient content. According to the WHO (World Health 
Organization), validation is a key step in the development of a nutrient profiling system. The aim was to assess the construct validity 
of five European nutrient profiling systems (Choices, Keyhole, (AFSSA), European Commission (EC) system and Food Profiler). 
Construct validity was assessed for each of the five-selected nutrient profiling systems by testing whether healthy foods (that is, 
identified as eligible by the system) make healthy diets, and unhealthy foods (that is, non-eligible) make unhealthy diets, using diet 
modeling. The AFSSA, EC and FoodProfiler systems were identified as valid, but differences in their levels of permissiveness 
suggested some misclassified food products. The two other systems failed the construct validity assessment. Among these three 
systems, the EC system is the less demanding in terms of nutritional information, it would, therefore, be the easiest to implement 
for regulating nutrition and health claims in Europe. 

European Journal of Clinical Nutrition (2013) 67, 1003-1005; doi:1 0.1 038/ejcn.201 3.95; published online 22 May 2013 
Keywords: nutrient profiling system; construct validity; linear programming; diet modeling 



INTRODUCTION 

Nutrient profiling classifies individual food products according to 
their nutrient content. In the 1924/2006 European regulation, 
nutrient profiling is proposed as a tool for deciding whether a 
given food might be eligible or not for bearing nutrition or health 
claims. 1 One core aim of the regulation is 'to avoid a situation 
where nutrition or health claims mask the overall nutritional status 
of a food product, which could mislead consumers when trying 
to make healthy choices in the context of a balanced diet'. 
Report of a WHO (World Health Organization)/IASO (International 
Association for the Study of Obesity) joint technical meeting 
recently highlighted the importance of using validated nutrient 
profiling systems and listed a number of possible validation 
methods. 2 These include assessment of construct validity, that is, 
testing whether healthy foods (identified as eligible by nutrient 
profile system) make healthy diets, and unhealthy foods (that is, 
non-eligible) make unhealthy diets. The aim of the present study 
was to assess the construct validity of five nutrient profiling 
systems, using diet modeling with linear programming for 
designing healthy and unhealthy diets. 3 



MATERIALS AND METHODS 

The dietary data and the food database were derived from the INCA 
French dietary survey. 4 Each food of the food database (n = 597) was 
allocated to one of two classes, eligible and non-eligible, according to each 
of the five European nutrient profile systems tested. Their main 
characteristics can be found in Table 1. Two systems are currently used 
for food labeling, namely Choices v2.2 5 and Green Keyhole, 6 two have 
been proposed to assess food products' eligibility to claim but that are 
not currently in use, namely the French AFSSA system (Agence Francaise 
de Securite Sanitaire des Aliments) 7 and the EC system (European 
Commission draft), 8 and FoodProfiler, a system in use for self-limiting 
advertisement to children. 9 



Using a previously described diet modeling approach, 3 the feasibility 
of designing healthy or unhealthy diets with eligible foods only, or with 
non-eligible foods only, was tested for each nutrient profiling system. 
Healthy diets were defined by the fulfillment of a set of a forty nutrient 
constraints, and unhealthy diets by the nonfulfillment of the same set of 
nutrient constraints. 3 To ensure realism to the modeled diets, constraints 
on foods and food groups were included and the possibility to design a 
2000-kcal diet was tested 3 Four feasibility tests were used to assess the 
construct validity of each system. Tests A, eating healthily with eligible 
foods only, and D, eating unhealthily with non-eligible foods only, were 
used to assess whether the systems respected the nutritional common 
sense. Tests B, eating unhealthily with eligible foods only, and C, eating 
healthily with non-eligible foods only, assessed more finely the level of 
permissiveness. The following terminology is used: 'Strict unfeasibility' 
means that no mathematical solution can be found at all. When solutions 
can be found, the range between the minimum and the maximum energy 
achievable is called the 'energy range'. When the energy range includes 
2000 kcal, the model is considered feasible. When the energy range 
excludes 2000 kcal, the model is considered unfeasible. A system is 
considered as valid when tests A and D are feasible, and tests B and C are 
unfeasible. 



RESULTS 

The percentage of eligible foods varied between systems: Choices: 
31 .8%; Green Keyhole: 35.6%; AFSSA: 35.8%; EC system: 46.5%; and 
FoodProfiler: 49.5%. 

Figure 1 shows that all tested systems allowed 2000 kcal healthy 
diets with eligible foods (test A), and 2000 kcal unhealthy diets 
with non-eligible foods (test D). 

Regarding test B, all tested systems exclude 2000 kcal from their 
energy range and were, therefore, considered as unfeasible: strict 
unfeasibility was observed for Choices and Green Keyhole 
systems. Unfeasibility was observed also for the FoodProfiler and 
EC systems, the maximum energy achievable being 1772 kcal and 
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Table 1. Main characteristics of the five European systems studied 


Name of the system 


Field of application 


Nutrients to limit 


Nutrients to encourage 


Reference amount 


Calculation 


Number of categories 


Choices v2.2 


1 u i- 

Labeling (In use) 


Added sugars, 


Fiber 


1 nn n 
i uu g 


I hreshold 


> ZD 






orrA, in a, ditiyy 










Green Keyhole 


Labelling (in use) 


Added sugars, 


Fiber 


100g 


Threshold 


>25 






SFA, Na 










AFSSA system 


Claim 


Added sugars, 


Fiber, Protein, Vitamin 


100 kcal 


Scoring 


Across the board 




Eligibility (draft) 


SFA, Na 


C, Vitamin D, Ca, Fe 


100g 






EU system 


Claim 


Added sugars, 


None 


100g 


Threshold 


13 




Eligibility (draft) 


SFA, Na 










FoodProfiler 


Advertising to 


Total fat, Added 


Fiber, PUFAs, Ca, Fe 


100 kcal 


Scoring 


Across the board 




children eligibility 


sugars, Added 












(in use) 


Na, SFA, TFA 










Abbreviations: AFSSA, Agence Francaise de Securite Sanitaire des Aliments; EC, European Commission; SFA, saturated fatty acids; TFA, trans fatty acids. 



Healthy diets with eligible foods 
(expected feasible) 



Choices v2.2 
Green Keyhole 
FoodProfiler 
AFSSA 
EC 



Unhealthy diets with eligible foods 
(expected unfeasible) 



Choices v2.2 
Green Keyhole 
FoodProfiler 
AFSSA 
EC 



(strictly unfeasible) 

I I 
(strictly unfeasible) 



2000 



4000 



6000 



8000 



Healthy diets with non-eligible foods 
(expected unfeasible) 



Choices v2.2 
Green Keyhole 
FoodProfiler 
AFSSA 
EC 



I I 
(strictly unfeasible) 

I 



-r- 



D Unhealthy diets with non-eligible foods 

(expected feasible) 

I 



Choices v2.2 •■ 



Green Keyhole 
FoodProfiler 
AFSSA 
EC 



2000 



6000 8000 



Figure 1. Energy range, in kcal, to design healthy/unhealthy diets with eligible/non-eligible foods. Test A assesses whether it is feasible to 
design healthy diets with eligible foods (it is expected to be feasible). Test B assesses whether it is feasible to design unhealthy diets with 
eligible foods (it is expected to be unfeasible). Test C assesses whether it is feasible to design healthy diets with non-eligible foods (it is 
expected to be unfeasible). Test D assesses whether it is feasible to design unhealthy diets with non-eligible foods (it is expected to be 
feasible). 'Strictly unfeasible' means that no mathematical solution can be found at all. When the energy range includes 2000 kcal, the model is 
considered feasible. When the energy range excludes 2000 kcal, the model is considered unfeasible. 



1801 kcal, respectively, and for the AFSSA system, with a narrow 
and unrealistic energy range (587-907 kcal). 

Regarding test C, strict unfeasibility was observed for 
FoodProfiler only. Test C was feasible for Choices and Green 
Keyhole (2000 kcal healthy diets allowed with non-eligible 
foods) and it was unfeasible for AFSSA and EC systems (the 
energy range for test C excluded 2000 kcal but the minimum 
energy achievable was close to 2000 kcal: 201 8 kcal and 
2162 kcal, respectively). 



DISCUSSION 

One important result is that, with all tested systems, it was possible 
to design healthy diets with eligible products and unhealthy diets 
with non-eligible products (tests A&D), demonstrating that all 
systems satisfied the nutritional common sense. In line with the 
latter results, Choices was previously presented as valid based on its 
beneficial impact on dietary quality in modeling studies simulating 



the replacement of foods not complying with the Choices criteria by 
existing Choices-compliant products. 10 Nevertheless, Choices and 
Green Keyhole allowed to design 2000 kcal healthy diets with non- 
eligible products, and were, therefore, identified as invalid according 
to this construct validity assessment. In contrast, FoodProfiler, AFSSA 
and EC systems were identified as valid because they answered as 
expected to the four tests. 

The three valid systems showed differences in their level of 
permissiveness. With the AFSSA system, it was almost feasible to 
design a 2000-kcal healthy diet with non-eligible foods (test C), 
which suggests that some healthy foods are misclassified as non- 
eligible. Therefore, this system rather display a 'keep the best' 
approach when classifying the unhealthy food products. In the 
opposite, with the FoodProfiler and the EC system, it was almost 
feasible to design a 2000-kcal unhealthy diet with eligible foods 
(test B), which shows that these systems rather display a 'stop the 
worst' approach by misclassifying some unhealthy foods as 
eligible. Analyzing the foods selected by linear programming in 
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these tests could help identify the misclassified foods, and then 
fine-tune the systems. 

The validation method used in the present study is perfectible, 
for instance by improving the realism of diets using more 
sophisticated diet modeling techniques. 11 Nevertheless, this 
study revealed that some nutrient profiling systems, among 
those currently used, show weaknesses by failing this construct 
validity assessment. 

Among the three systems identified as valid in the present 
study, the EC system is the less demanding in terms of nutritional 
information, it would, therefore, be the easiest to implement for 
regulating nutrition and health claims in Europe. 
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